Skip to content

Right-size CI runners onto Blacksmith (FIG-518)#131

Merged
SamGalanakis merged 1 commit into
mainfrom
samuel-fig-518-runners
Jul 25, 2026
Merged

Right-size CI runners onto Blacksmith (FIG-518)#131
SamGalanakis merged 1 commit into
mainfrom
samuel-fig-518-runners

Conversation

@SamGalanakis

Copy link
Copy Markdown
Collaborator

Only 3 of 18 CI jobs used Blacksmith; the rest sat on ubuntu-latest, including the four slowest remaining jobs (the Confidence-fast shards at 8m/8m/6m/5m) which dominate the ~9 min critical path.

FIG-518 had this deferred on two claims from 2026-07-21 — that blacksmith-4vcpu was unprovisioned for this org, and that supply arrived in unpredictable bursts. Both were re-tested empirically and are no longer true. An unprovisioned label queues forever rather than erroring, so this had to be measured: a probe ran one job on every size, all five concurrently in a single matrix, and all were allocated within 9–10 seconds with correct cores/memory (2/7Gi, 4/15Gi, 8/30Gi, 16/62Gi, 32/113Gi). Tiered sizing is therefore available, which the old "16vcpu or nothing" constraint ruled out.

Sizing

job measured runner
confidence-fast (fault-matrix, minimizer-fixtures, scenario-harnesses, sim-generated) 8/8/6/5m 16vcpu
test-doc 5m 16vcpu
lint, stack-budget, E2E agent-workbench 2/4/5m 8vcpu
postgres-store, E2E agent-service 3/2m 4vcpu
E2E workflow-graph-roundtrip, confidence-fast-summary 1/0m 2vcpu

Deliberately NOT migrated

sim-unit-perf-guards and perf.yml stay on ubuntu-latest. They enforce scripts/perf_guard_budgets.json, which is calibrated to that machine class — moving them would not merely make them faster, it would make every budget pass trivially and hide real regressions. Those budgets were just found to have been ungated for two days (PR #128); silently invalidating them again would repeat the same failure. It is kept on ubuntu via a per-matrix-entry runner value rather than splitting the job, so no job or matrix name changes — the ~20 required check names are byte-identical, so no required gate can be silently dropped.

cross-check is untouched (matrix includes macOS; no Blacksmith darwin).

Runner behaviour is only provable in CI. Per FIG-518's second hard-won rule, this wants observing under concurrent runs (a main merge overlapping an open PR) before being called good — a single uncontended green run measures machine speed, not whether CI works.

Release-Notes: Internal: Right-size merge-gate CI jobs on Blacksmith runners and pair migrated jobs with Blacksmith's Rust cache.
@SamGalanakis
SamGalanakis merged commit 5a873be into main Jul 25, 2026
20 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant